On Integration Repositories, Build Sheriffs, and Patch Backouts

If a developer commits a patch that breaks the build (i.e., that does not compile or causes tests to fail), the given patch should be backed out in order to keep the repository stable while the developer writes a fix. In 2011 Firefox adopted rapid release cycles and changed its code integration model. Previously, developers committed code to the central repository off which patches branched; in 2011, they started committing to a separate integration repository. Now, only patches that successfully build are merged by dedicated build sheriffs into the central repository. This liberates developers from needing to perform comprehensive testing prior to committing patches [3]. Khomh et al. [1] studied Firefox’s process changes and concluded that bug fix patches were released quicker after 2011. Mäntylä et al. [2] showed that Firefox’s rapid releases left less time for manual testing, which became focused on specific areas. This study also observed Firefox’s changes. Analyzing commits for 41,305 issues from 2009 to 2013, we determined that the proportion of issues with commits backed out because of broken builds increased from 3.5% (2009–2011) to 8.3% (2011–2013). This is a result of less comprehensive developer testing and sheriffs backing out broken patches. This increase in backouts, however, is a non-issue. Under the new process, those backouts are performed in the integration repository prior to merging. The central repository became even more stable: in the same period, the proportion of issues with commits backed out later, during manual testing, dropped from 3.1% to 1.5%. BODY Integration repositories and build sheriffs allow developers to test less comprehensively while keeping the main repository stable.